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Classroom teachers on all., levels need more awareness of readability 
and readability formulas. They need to better understand how to use 
and interpret the formulas and how to avoid misusing and overrelying 
on them. They need awareness of the limitations of readabi 1 i ty formul as 
and the factors affecting readability that are not "accounted for by the 
formulas. These, factory we will refer to as the n black box." Teachers 
need to realize the impact on educational materials and perhaps on 
learning outcomes that result from using readability formulas and what 
the alternatives to formulas might be. The purpose of this article Is 
to help provide some awareness of these issues. 

> ' / 

' Interest in Re adability Formulas 
, — — ■ » 

The increase in interest and use of readability formulas is due to 
many influences. "There is more sensitivity now on the part of. the 
(federal government to the jargon and bureaucratese, legalese^ and 
technical styles of writing that cause reading problems for the general 
public. Increased interest of legislators, school boards and teachers 
from all levels is no doubt related to the recent concern with basic 
skills and the decline in reading skills. The declining school 
enrollment has made publishers of educational materials more competitive 
and the use of readability data is used as evidence by publishers that 
one textbook is superior to another. 

Reading researchers are also interested and concerned, so reada- % 
b 1 1 i ty research, is flourishing. Many researchers are doing predictive * 
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readability research, trying to improve existing formulas or devise new 
ones that will better predict the difficulty level of texts. Other \ 
researchers are doing productive readability research, trying to come 
up with guidelines .for writers to produce more readable writing. Teachers 
will be increasingly concerned with the findings of both predictive 
and productive readabi 1 » ty research as they face ever-widening spreads 
of reading abi li ty- because of ma instreaming programs for the gifted 
and handicapped and. open enrol lment 'on the college level. 

e Readabi 1 i ty Defined 
The term 'readability' has different meaning^ for different people. 
It can mean legibility, the reading base, the interest value of a 
text or the comprehension base result ing. from the style of writing. 
Chaljl (1958) gives a more complicated, interactive view of readabi 1 i ty : 
'The idea underlying readability measurements is the appropriate 
matching of reader and printed material'.' (p. 9)- The question can 
be raised whether actual practice has kept up with this sophisticated 
definition. Does appropriate matching of reader and printed materials 

result from the rise of readability formulas? 

/ ' . - • • 

; There is no doubt that teachers do want to match reader and texts 
appropriately or that the formula developers wanted to help teachers / 
do just this. The intentions of the formula developers were to help 
teachers and help children learn the content of the text and give them 
a. better reading experience, one that is less frustrating. • For a long 
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time teachers had to do the matching based on their subjective judgments, 
without benefit of the . formulas , if they did matching at all. Neither 
teachers nor publishers were interested in measuring readability until 
fairly recently, so formulas, according to Fry (1980), are better than 
noth ing-'and are a step forward. 

Readability Formula Problems 
Perhaps formulas are a step forward since they are more objective 
than teacher judgments, but because the most widely used formulas are 
based on just three variables — vocabulary, familiarity, word length, 
and sentence length, they have serious 1 imi tat ions^^Tfie problem with 
the readability formulas is that they do not use as variables the 
important "black box" factors: content and the reader's prior knowledge 
of it; format; organization; text structure; content; style; flow of 
information; lack of .conflict structure; idea and inference density; 
grammatical and psychol ingui stic factors; and reader characteristics. 
We will return to these "black box" factors a little later. 

Readability Formula Assumptions 
The applicability of readability formulas, rests on several basic 
assumptions that users should recognize (Bruce & Rubin, 1980). These 
assumptions are that : 

1. The test is honestly written — that it is written not to 
satisfy the readability formula, but rather to satisfy some other com- 
municative goal. 



2. Material may be freely read—that there are no time constraints. 

3. Higher-level text structures are irrelevant. (The formulas 
do not take into account organizational materials, information about 
intentions and goals.) 

A. Purpose in 'reading is irrelevant. 

5. Statistical averages are meaningful for individual cases. 
(Formula use implies that statistical averages regarding .both texts and 
readers can provide useful informat ion regarding the appropriateness 
of an individual text for an individual person;) 

(. 6. Readers you are interested in are the same as the readers on 
Whom the readability formula was validated. (Attempts to expand the 
u se o f^the^ormulas to evaluate. materials for readers whose background, 
dialect, and purpose in reading differs from the readers used in validation 
is likely to lead to difficulties.) 

7.4 It is sufficient to eva1uate\the readability of a text in 
terms of lexical and syntact ic factors alone, ignoring such factors as 
pictures, graphic convention, rhyme, and genre. 

8. Language rather than conceptual content -is the main determiner 
of text diff icul ty. 

" Educators and publishers need to ask themselves what effect these 
assumptions have on students and educational materials when readability 
formulas are used for adaptions, selections of texts for readers of 
different cultural backgrounds, designing special'texts for children, 
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selection of test passages , choos ing trade books, television captioning, 
and designing remedial readers. ".It is very probable, as Bruce and Rubin 
point out that because of these assumptions, readabi 1 i ty formula use is 
actually restricted to trivial cases of little importance for educational 
or social pol icy. 

i Bradley and Ames (1978) have listed an additional six (6) readability 
formula assumptions that are particularly important for classroom 
teachers using basal readers. 

1. The content of a basal varies little in readability. This 
assumotion is not supported since the authors found in their study of 
thirteen (.13)^ reading level range—from first grade to college in 
Houghton-Mifflin's Kaleidoscope basal. 

2. Readers within a basal series are graded from easy to difficult. 
This is generally true in terms of averages, but a substantial part 

of the third reader was more difficult than a substantial part of the 
fourth reader in the HoughtpnHMiff i in series. 

3. Stories wi thin a' basal reader differ little in difficulty from 
one to another. The authors found in their study that stories in the 
same book differed in average readabi 1 i ty as well as the range of 
readability and that the difference in readability from one story to 
another became more extreme as the books increased in overall difficulty. 

4. Stories near the beginning of a basal reader are generally 

less difficult than those at the end. This assumption was not supported. 



A story at the beginning of Kaleidoscope had an eighth grade readability 
level according to the Fry f ormula wh i 1 e a story near the end was at a 
second grade level of difficulty. "7 

5. A good estimate of the readability properties of a basal reader 
may be obtained, by using readability information based on a few randomly 
drawn book samples. The authors found that 2k samples needed to be 
drawij before there was any consistent agreement with>the book level. 
Fitzgerald "(1980; 1 98 1 ) found essentially the same situation to -be true 
for secondary textbooks and basal workbooks. 

6. A student's instructional reading level (IRL) within a basal 
reading series can be' predicted by using a traditionally constructed 
informal reading inventory. This assumption was found urisupportable 
because of the extensive intrabook readability variation. 

Clearly, teachers must be very careful and cautious about judging 
basals according to the number the publishers place on the cover. 

. . ■ i Black Box Factors 

One of the most important factors not considered as a variable in 
the readability formulas is the content of the text—the conceptual 
difficulty and the reader's prior knowledge of the concepts and content 
of the text. For students reading the McCal 1-Crabbs passages (used 
to validate the readability formulas) on a specific reading level, 
there would be wide variatfon in prior knowledge and content difficulty 



in a passage for instance about winters in Vermont and the testing of 
the atom bomb,. Diablo, or a passage about the coastguard activities 
and one about jet streams. The lexical and syntactic variables would 
not give a true indication of the readabi 1 i ty of these passages. 

^Format is another .factor. Some .formats are more familiar to studen 
.than others and thus more readable, and some are by their nature more 
difficult to read and comprehend. Fairy tales and short stories are 
no, doubt on a different readability level than math story problems and 
argumentative essays. A literary description of a haunted house would 
be on c a different level than a technical description of a tool or a ^ 
real estate ad. Genre and discourse types vary considerably in reada- 7 
bility levels. The organi zat ion of a text makes a difference in 
readability, too. Texts can be organized inductively or deductively, 
spatially, temporally, climatically, all of which makes a difference 
in the readability level of texts. 

Recent research by Kintsch and Vipond (1977) suggests some other 
factors that are "part of the ."black box" of readabijl i ty . Looking at 
text structure which involves higher-level text features, these 
researchers analyzed passages that had been measured with readability 

i ■ / .. ' • .. . 

formulas and found that -some which were predicted^tq be easy were in 
fact difficult to process because of these factors: 

1. number of propositions in a text (density of ideas). 

2. number of different arguments. - 



.8 ' 

•3. number of coherence networks. 

k. number of inferences required. < - 

5. number of long-term memory searches and reinstatements of 
propositions jnto short-term memory. v 

6. number of reorganizations required to arrive \at the best- 
organized text base. . 

^ These researchers believe that readability must- be theory-Based and 
that readability formulas are atheoreti cal . Investigating readability 
within the framework of their text analysis model,! they suggest that 
the concept of readabi 1 i ty is beyond: salvat ion. Readabi 1 ity i s not some- 
how an inherent property of texts, but lis the result of the interaction 
between a particular text (with its text characteristics) and particular 
readers (with their Jnfonriatioh-processing characteristics). Readability 
must be defined for specific texts and specific readers and the single 
readability score replaced with a readability profile that shows how a 
particular text would be responded to by different readers. Kintsch 
and Vipond suggest that such measures as reading time, amount recalled, 
and number of questions answered correctly are better indicators of 
readability than formulas, when these measures are theory dependent. 

* Readability formulas are not sens i tive to context ; some contexts 
make a word easy to understand and other context make the same word 
•'difficult to understand — metaphors for example. The word fami 1 iar ity 
variable is. a problem, too, since a single measure is used. What is 

\ 

\ 
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not considered in toord ..,fami 1 Sari ty 'is the context the word is used in, 
the difference between oral and written familiarity,, and the background 
of the reader. " . 

Style is also part of the "black box. 11 There are styles that suit 
and styles that do not. Readers find texts readable or not because of 
an author's style--h is manner of expression and the tone or mood of 
the text. Whether a reader is put off or attracted to a text, considering 
it nonreadable or readable, depends not only on text structure, format, 
organization, and content, but also on style and tone. 

Discourse analyst Vonda Kopple (1980) is critical of Readability 



formulas because they , do. not consider flow of information. Thg 



relation- 
ships between sentence topics is a very important factor of readability, 
for expository prose paragraphs with identical or related sentence 
topics are easier to read. Bruce and^Rubi n (1980) are criticaj of 
basal 1 stories constructed to conform to readability formula constraints, 
for they lack coherence and ignore other crucial characteristics of 
texts. The basal sanctions are hard to read because there is no 
familiar structure of conflict and resolution expectation and because 
of the lack of conflict, suspense, purpose, humor and point of view. 
Remedial reading texts are another of their concerns. Readability 
formulas are used in designing and choosing remedial reading texts, 
but. this approach fails to take account of certain problems. In these" 



-texts, maintaining a coherent story line is less important th.an 



introducing the written forms of particular sounds such 1 as o and £. The 
higher level structures are ignored since the text is written just to 
do the teaching function, thus making thqm less readable unnatural: 
texts. 

Although t^xt structure factors are very much a part of the "black 

box, 11 so are' gramma ti ca! factors. The results of a study by Charrow 

'. - ' ' * 
(1980) indicated that grammatical complex i ty is at the root of the diffi 

culties with legalese. For instance, when jury instructions were 

rewritten, comprehension for; the jurors increased, even though the 

Flesch readability formula had predicted the original instructions 

should be readable. Readability formulas with thei r vari ables of 

vocabulary f amM iari ty , word length, and sentence length, and in the 

caseof the Flesch formula, the additional variable of second person 

pronouns, do not consider grammatical complexity. Yet, readability is 

I 

affected by unusual placement of phrases, certain grammatical con- 
structions such as the a£ to construction, multiple negatives, 

V ■ 

nominal izatidns, "strings of adjectives or lists of nouns, sorae^pass 1 ves , 
and paraphrases of ideas, etc. When a text has grammatical complexity, 
making it more readable requires increasing paragraph and sentence 
length by adding more context for clarity. Readability formulas, 

however, would give the rewritten text a higher reading level because 

1%, 

of the increased sentence length. 

Since reading is- a personal, interactive process, the text itself * 
is not the only thing to consider for readability. Bruce and. Rubin 



( 1 980) mention eight factors that can affect readability for a student: 
). How do I feel — tired? hungry? .eyes hurt? distracted? 
'\ preoccupied? 

2. HoW interested am I in the topic or story? 

3. What do I know about the subject — what is my background 
knowledge? . 

k. How similar is the writer's language to mine? 

5. How plausible do I think the author's beliefs, presuppositions 
are? What do I take for granted? 

6. Why am I reading this? for facts? general knowledge? escape? 

7. How long -do I have to read this? How does this affect my 
reading goals? 

8. What do I want to do with the information? \ 
Clearly these reader characteristics^ as well as task and text 

characteristics are important factors for. ascertaining readability and 
^consequently are part of the "black box." 

Other Readab i 1 i ty Formula Limitations 
Teachers certainly need to be aware of the "black box" factors not 

considered in readability formulas^and the assumptions they are based 

\ " ' - 

on. But there are other problems wi th readabi 1 \ ty formulas. The 

sampling procedure problem has been menti^i^ed^^^rousjV. Another is 

the validity problem. Validation studies have b'een<unbel ievably weak 

and circular in nature, validating one. formula against another^and 



measuring simply what is taught in the basal readers o rather than real- 
world literacy. The McCal 1-Crabbs criterion passages are dated in regard 
to both materials and norms and are quite similar to those in basal 
readers. The formulas are based' on elementary level passages but are 
supposed to be predi cti ve up. to adult reading levels. The use of 
cloze tests as a measure of comprehens i bi 1 i ty is questionable since 
the psychological prpcesses are not the same in cloze tests as the ones 
in reading a text. Also there is great variability among the formulas 
in estimating grade level; with some texts there can be a four grade 
level difference between one formula and another. The readability 
formula scores dre imprecise and unreliable (Seldon, 1980; Fry, 1980). 

Chall (1977; 1979) alsoisoynds a note of caution about the use of 
readability formulas for matching books and' readers. • None of the 
readability formulas tells how difficult reading materials should be. 
They seem' to have been used mainly to write, edit, a nch select easier 
books. The whole issue of how best to match the readability of a text 
to the ability of a reader for optimal reading comprehension and 
learnings wi dely ''"d iscussed but unresolved. A recent study of text- 
books in relation to'declining SAT' scores found that the decl ine was 
associated with a decline in diff icul ty^of /textbooks over a 30-year 
period and. that^a recent increase in difficulty in primary school 
texts was associated with rising read ing< scores. This suggests that 
te^ts Yhat^ard" too easy may produce lower scores on verbal tests and 
may be detrimental to the development of high. level reading and 



interpretation skills. The concern should go, it seems, not only to 
books that may be too difficult but (to those that may be too easy. 

In spite of all this, the preference for easy textbooks is still 
strong. Publishers claim that teachers of the elementary grades are 
requesting science and social studies textbooks two years below the 
grade placement of the children. MacGinitie (.1981) points out that 
there are two ways to make text easier to read. One is to eliminate 
text characteristics that cause many students difficulty. The other i 
to give students help and experience in understanding text that has 
those characteristics. There are also two ways to make texj: difficult 
to read. One is to. include "text characteristics that cause many 
.Students -difficulty. The other is to insulate students from help and 
experience in reading text with various characteristics. MacGinitie 
would like teachers to realize that choosing short-sentence versions 
of materials for students because the students are accustomed to 
reading that sort of text is the wrong way to go. Instead, teachers 
should, choose mater ial where ideas are easier to understand because 
sentences are longer and make use of sentence structure to specify, 
relationships. The materials that,, teachers give students to read 
determine in large part, current reading abilities and readability 
findings. Green ( 1 98 1 ) agrees and proposes abandoning al 1 /adaptions 
in basals and using age appropri ate natural texts since the current 
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procedure for "s imp 1 i fy ing"\texts result in making texts unnatural, more 

difficult to understand, and less . interesting than the. or i g inal s . 

' / 

Concl us i on 

In the future we must fit readability formula research and findings 

more completely into the 1 arger p icture of reading behavior and human 
behavior more general Ty. We need different formulas for different 



users. Classroom teachers doing gross screening can use a simpler 
formula; publishers doing extensive measuring and textbook adaption 
committees need accuracy. and a sophisticated complex formula. A 
scientific approach is now necessary for. readabi 1 i ty formulas rather 
than the earlier practical approach (Klare, 1979). And "a clearer : 
^theoretical approach to readability is mandatory (Kintsch S Vipond, 
1977). Researthers must take the "black .box 11 factors , into account. 
Support of readabi! ity research and application must come not only from 
universities and the government but also from the book Industry, 
particularly the textbook sector (Chall, 1979). Readabi 1 i ty, formula 
research must be an interdiscipl inary endeavor. Linguists, psycho- 
linguists, socio-1 inguists , \cognit ive psychologists, discourse analysts, 
rhetoricians, and reading educators must all work together to determine 
what constitutes the "black box" of readability. 



And teachers must us'e readabi 1 i ty forraul as with understanding and 1 
caution.. It is obvious th^at wh^i le formul as seem to provide simple j' 
ways of measuring readability, -^hey are only a place to begin. "Teacher: 
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need to have some idea of their students', background, knowledge, ways of 
thinking^ present educational level, their vocabulary use, their 
familiarity with other writing, because all that makes a difference in 
how hard it is to read a passage," notes Fry (1980). In other words, 
teachers" must become aware of the many factors that make up that "black 
box" of readability and rely more on the i r. own informal subject ive 
judgment in matching readers and texts. 
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